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Abstract 


This  report  summarizes  the  results  of  the  Capability  Maturity  Model  Integration  (CMMI) 
Interpretive  Guidance  Project.  It  summarizes  and  analyzes  the  7500  comments  received  re¬ 
garding  CMMI  adoption  that  were  reported  by  CMMI  users  and  potential  users.  It  also  de¬ 
scribes  the  actions  being  taken  by  the  Software  Engineering  Institute  (SEI)  to  address  the  is¬ 
sues  identified  by  Interpretive  Guidance  Project  participants. 


Although  the  initial  goal  of  the  project  was  to  develop  interpretive  guidance,  after  data  gath¬ 
ering  and  analysis  the  team  realized  that  most  respondents’  input  did  not  require  interpretive 
guidance.  Based  on  a  relatively  small  number  of  comments,  interpretive  guidance  was 
planned,  including  papers,  frequently  asked  questions  (FAQs),  and  new  CMMI  courses. 
However,  participant  comments  clearly  showed  CMMI  interpretive  guidance  to  be  less  of  an 
adoption  issue  than  suspected. 


Some  comments  covered  issues  already  being  addressed  as  part  of  SEI  activities,  including 
the  development  of  Standard  CMMI  Appraisal  Method  for  Process  Improvement 
(SCAMPIsm)  Class  B  and  C  methods,  the  collection  of  cost  and  benefit  information,  and  the 
creation  and  improvement  of  CMMI  training  courses.  The  majority  (approximately  80%)  of 
the  actionable  comments  received  were  best  handled  as  change  requests  to  help  guide  the  on¬ 
going  improvement  of  the  CMMI  Product  Suite. 


SM  SCAMPI  is  a  service  mark  of  Carnegie  Mellon  University. 
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1  Introduction 


Since  the  release  of  Version  1.1  of  the  CMMI  Product  Suite  in  January  2002,  the  SEI  and  its  Part¬ 
ner  Network  have  been  helping  organizations  understand  and  adopt  CMMI.  The  CMMI  Interpre¬ 
tive  Guidance  Project  was  formed  to  understand  how  CMMI  is  being  adopted  and  used  by  soft¬ 
ware,  IT,  and  IS  organizations.  This  report  and  its  companion  report,  CMMI  Interpretive 
Guidance  Project:  Preliminary  Report,  summarize  the  comments  collected  and  describe  the  ac¬ 
tions  being  taken  to  address  them  [Chrissis  2003a], 


1.1  Lessons  Learned 

When  the  Interpretive  Guidance  Project  was  initiated,  project  members  expected  that  their  re¬ 
search  would  identify  problem  areas  that  could  be  addressed  by  specific  interpretive  guidance. 
Team  members  expected  to  find  that  particular  process  areas  or  practices  would  be  problematic 
for  software,  IT,  and  IS  organizations.  Team  members  imagined  that  by  providing  more  informa¬ 
tion  to  help  these  organizations  use  parts  of  CMMI,  it  would  ease  the  adoption  struggles  of  many 
organizations. 


However,  the  data  collected  generally  did  not  confirm  these  expectations.  At  the  conclusion  of  the 
Interpretive  Guidance  Project,  the  team  members  realized  that  the  comments  they  gathered  did 
not  logically  conclude  a  strong  need  to  develop  interpretive  guidance  but  rather  provided  com¬ 
ments  that  are  summarized  in  the  following  sections. 

1 .1 .1  Positive  Comments 

Instead  of  identifying  serious  problems  that  could  be  addressed  by  interpretive  guidance,  the  team 
received  many  comments  insisting  that  CMMI  was  useful  without  interpretation.  Much  of  the 
feedback  received  was  positive.  The  team  hadn’t  expected  that.  After  all,  it  was  looking  to  target 
the  problem  areas  hampering  the  adoption  of  CMMI  in  software-related  industries.  Although 
there  were  many  suggestions  for  improving  CMMI,  ultimately  no  fundamental  flaws  emerged. 

1 .1 .2  Minimal  Need  for  Guidance 

Of  all  the  data  received,  only  about  20  percent  pointed  to  the  need  for  interpretive  guidance  and 
many  of  these  comments  were  countered  by  comments  from  organizations  that  found  little  diffi- 
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culty  with  CMMI.  So,  instead  of  creating  interpretive  guidance  as  the  project  name  suggests,  the 
team  converted  the  majority  of  comments  to  change  requests  for  CMMI  Version  1.2  development. 

1 .1 .3  Software  Not  Unique 

Although  the  questions  posed  by  the  Interpretive  Guidance  Project  were  aimed  at  software,  IT, 
and  IS  organizations,  responses  were  received  from  a  variety  of  organizations  besides  those  tar¬ 
geted.  The  comments  from  these  organizations  were  similar  to  and  consistent  with  the  comments 
from  the  software,  IT,  and  IS  organizations.  Largely,  these  other  organizations  represented  sys¬ 
tems  engineering  and  acquisition.  This  aspect  of  the  feedback  seemed  to  imply  that  adoption  dif¬ 
ficulties  were  not  specific  to,  unique,  or  limited  to  software,  IT,  and  IS  organizations. 

1.1.4  Activities  Underway 

Because  the  Interpretive  Guidance  Project  was  a  two-year  project,  as  the  team  collected  com¬ 
ments  it  could  forward  its  analysis  of  these  comments  to  those  responsible  for  new  and  existing 
SEI  activities  intended  to  help  CMMI  adoption.  So,  it  was  no  great  surprise  that  when  the  team 
finished  its  analysis  it  found  that  many  of  the  issues  identified  were  either  already  being  ad¬ 
dressed  by  existing  SEI  activities  or  by  new  activities  being  planned,  including  the  20  percent  in 
which  interpretive  guidance  was  suggested. 


1.2  Resulting  Actions 

Generally,  it  appears  that  CMMI  adoption  is  going  well  for  those  who  have  chosen  to  do  so.  Most 
organizations  judged  that  CMMI  was  adequate  for  guiding  their  process  improvement  activities 
and  that  their  prior  investments  in  process  improvement  helped  them  to  adopt  CMMI.  Neverthe¬ 
less,  the  actionable  comments  that  identified  issues  required  action. 

The  actionable  comments  provided  important  input  to  CMMI  Product  Suite  Version  1.2  devel¬ 
opment,  to  various  SEI  activities,  and  to  CMMI-related  research  and  communication.  The  great¬ 
est  number  of  comments  (2,200)  was  converted  to  CMMI  change  requests  to  be  used  in  the  de¬ 
velopment  of  CMMI  Version  1.2. 

As  a  result  of  the  Interpretive  Guidance  Project  Team’s  analysis  of  the  comments,  it  found  that 
there  were  three  main  areas  where  interpretive  guidance  was  needed: 

1 .  guidance  about  CMMI  model  components  not  found  in  legacy  models 

2.  guidance  for  addressing  overlaps  within  CMMI,  specifically  the  overlap  between  generic 
practices  and  process  areas 

3.  guidance  for  implementing  generic  practices 
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Interpretive  guidance  will  be  developed  in  these  areas  and  will  be  packaged  appropriately,  typi¬ 
cally  as  part  of  documents  planned  by  projects  already  underway.  For  example,  additional  guid¬ 
ance  about  implementing  generic  practices  will  be  added  to  the  CMMI  Frequently  Asked  Ques¬ 
tions  on  the  SEI  Web  site. 

Details  of  the  Interpretive  Guidance  Project  Team’s  activities  and  findings  are  described  in  the 
remaining  sections  of  this  report. 
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2  The  Project 


Although  the  focus  of  this  report  is  on  the  comments  received  by  the  Interpretive  Guidance  Pro¬ 
ject,  it  is  useful  to  understand  why  and  how  the  project  was  initiated.  The  CMMI  Interpretive 
Guidance  Project  is  best  understood  within  the  context  of  CMMI  history. 

The  Interpretive  Guidance  Project  began  as  a  response  to  input  the  SEI  received  at  a  CMMI 
Workshop  in  which  some  participants  said  that  CMMI  was  difficult  for  software  organizations  to 
use.  This  workshop  was  held  May  7-8,  2002.  The  purpose  of  the  workshop  was  to  understand 
CMMI  adoption  barriers  and  benefits  for  software,  information  technology,  and  information  sys¬ 
tems  organizations.  A  special  report,  A  Report  on  the  May  2002  CMMI®  Workshop,  resulted  that 
summarized  the  workshop  discussions.4 

During  the  workshop,  there  was  considerable  discussion  about  what  exactly  the  software  commu¬ 
nity  needed  to  guide  its  process  improvement  efforts.  Possible  solutions  discussed  during  the 
workshop  included  the  following: 

•  maintaining  the  SW-CMM  indefinitely 

•  creating  a  “software -only”  version  of  CMMI 

•  developing  CMMI  interpretation  guidelines  for  software  organizations 

The  difficulties  identified  at  the  workshop  became  an  important  concern  because  the  SEI  sought 
CMMI  to  be  considered  an  upgrade  of  the  SW-CMM  by  users,  and  recommended  that  software 
organizations  that  were  beginning  CMM-based  process  improvement  efforts  choose  CMMI  as  the 
basis  for  their  improvement  efforts.  Therefore,  if  additional  help  was  needed  by  software  organi¬ 
zations  to  enable  them  to  adopt  CMMI,  the  SEI  wanted  to  provide  it. 

After  the  workshop,  there  were  several  actions  taken.  The  decision  was  made  to  continue  with  the 
sunset  of  the  SW-CMM.  A  “software -only”  version  of  the  CMMI  model  (CMMI-SW)  was  cre¬ 
ated  and  made  available  to  the  public.  And,  the  SEI  formed  the  CMMI  Interpretive  Guidance  pro¬ 
ject  to  research  how  best  to  help  software,  IT,  and  IS  organizations  adopt  CMMI. 

The  Interpretive  Guidance  Project  was  formed  to  find  out  just  what  kind  of  problems  were  being 
encountered  by  these  organizations  and  what  kind  of  help  was  needed. 


This  report  is  available  by  request  from  cmmi-comments@sei.cmu.edu. 
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The  objectives  of  the  Interpretive  Guidance  Project  were  defined  as  follows: 

•  Understand  and  address  the  issues  that  software  organizations,  with  a  special  emphasis  on 
commercial  software,  IT,  and  IS  organizations,  have  when  using  CMMI. 

•  Enable  current  Capability  Maturity  Model  for  Software  (SW-CMM)  users  to  more  easily 
upgrade  to  CMMI. 

•  Eliminate  as  many  barriers  to  CMMI  adoption  as  possible. 

•  Encourage  CMMI  adoption. 

An  Interpretive  Guidance  Project  Team  was  formed  to  perform  the  work.  An  Interpretive  Guid¬ 
ance  Expert  Group  was  also  formed  to  help  guide  the  work  of  the  project  team  and  to  help  ensure 
the  work  would  be  useful  and  objective.  The  team  did  not  presume  they  understood  the  problem, 
so  their  first  activities  were  focused  on  data  collection. 

In  the  next  section  the  data  collection  activities  of  the  project  team  are  described. 
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3  Data  Collection 


Data  was  collected  using  several  means:  (1)  facilitated  birds-of-a-feather  (BOF)  sessions  at  con¬ 
ferences,  Software  Process  Improvement  Network  (SPIN)  meetings,  and  selected  CMMI  training 
courses;  (2)  an  online  survey;  and  (3)  detailed  interviews  of  organizations.  Details  of  how  the 
BOF  sessions  were  conducted  as  well  as  the  detailed  results  of  these  events  and  the  online  survey 
are  covered  in  the  report  CMMI  Interpretive  Guidance  Project:  Preliminary  Report  [Chrissis 
2003a], 


3.1  BOF  Sessions 

The  primary  purpose  of  the  BOF  sessions  was  to  identify  the  areas  of  CMMI  that  required  addi¬ 
tional  guidance.  BOF  session  attendees  completed  a  background  questionnaire  to  help  the  Inter¬ 
pretive  Guidance  team  understand  the  context  from  which  interpretation  issues  would  arise. 

At  each  BOF  session,  attendees  selected  one  or  more  areas  of  interest  and  joined  a  working  group 
to  provide  their  input  and  discuss  CMMI  issues  with  other  attendees.  Generally,  each  working 
group  provided  a  short  presentation  outlining  their  group’s  top  five  issues.  The  sessions  ended 
with  a  general  discussion  period. 

The  following  artifacts  were  collected  from  each  session: 

•  flipcharts  describing  the  issues  discussed  by  the  working  groups 

•  detailed  notes  taken  by  volunteer  recorders  selected  from  within  the  working  groups  that 
documented  the  context  of  issues  and  other  relevant  information 

•  marked  up  “lean  view”  of  the  CMMI  Framework  collected  from  participants  to  capture  issues 
that  were  not  recorded  or  discussed  during  the  working  group  session  due  to  time  or  other 
constraints 

•  CMMI  background  questionnaires  that  described  participants’  background  and  context 

For  more  information  about  the  comments  received  from  the  BOF  sessions,  see  the  report  CMMI 
Interpretive  Guidance  Project:  Preliminary  Report  [Chrissis  2003a]. 
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3.2  Web-Based  Questionnaire 


Geography,  schedule,  and  other  barriers  kept  many  members  of  the  targeted  community  from 
providing  input  at  the  BOF  sessions.  Consequently  the  team  chose  a  Web-based  questionnaire  as 
an  additional  vehicle  for  collecting  information.  The  Web-based  questionnaire  allowed  the  team 
to  collect  in-depth  information  regarding  CMM1  adoption  and  transition  since  participants  could 
provide  detailed  information  without  the  time  constraints  of  a  workshop  session. 

Members  of  the  Interpretive  Guidance  team  and  the  SEI’s  Software  Engineering  Measurement 
and  Analysis  Initiative  created  the  Web-based  questionnaire.  To  capture  data  consistent  with  other 
data  gathering  tasks,  the  questionnaire  included  many  of  the  components  of  the  BOF  sessions. 
Any  input  to  the  questionnaire  that  could  identify  the  respondents  or  their  organizations  was  (and 
is)  held  in  strict  confidence  by  the  SET 

The  Interpretive  Guidance  Project  Team  invited  over  7,000  people  to  participate  in  the  question¬ 
naire.  Of  those,  668  elected  to  participate.  For  more  information  about  the  comments  received 
from  the  Web-based  questionnaire,  see  the  report  CMMI  Interpretive  Guidance  Project:  Prelimi¬ 
nary  Report  [Chrissis  2003a]. 


3.3  Detailed  Interviews 

The  Interpretive  Guidance  Project  Team  received  both  positive  and  negative  comments  from  BOF 
sessions  and  the  online  survey.  To  gain  more  detailed  information,  the  team  chose  to  interview  a 
select  group  of  respondents  to  determine  how  individuals  and  organizations  resolved  the  issues 
identified  by  the  earlier  interpretive  guidance  data  collection  activities.  This  information,  when 
combined  with  the  much  broader  data  collected  from  the  survey  and  BOF  sessions,  provided  a 
more  complete  picture  of  CMMI  adoption  to  help  guide  interpretive  guidance  activities. 

These  interviews  were  conducted  either  at  a  company  site  or  by  telephone.  Comments  from  these 
interviews  were  held  in  the  strictest  confidence  and  were  subject  to  the  same  high  confidentiality 
and  non-attribution  guidelines  that  governed  the  Web-based  questionnaire.  In  both  cases,  com¬ 
ments  were  presented  only  in  summary  form  or  in  a  way  that  could  not  be  attributed  to  any  par¬ 
ticular  person  or  organization. 

From  a  candidate  list  of  approximately  forty  organizations,  the  Interpretive  Guidance  Project 
Team,  with  guidance  from  the  Interpretive  Guidance  Expert  Group,  selected  eight  organizations 
to  interview.  This  set  of  organizations  represented  a  cross-section  of  the  community  of  interest 
and  had  addressed  many  of  the  issues  discovered  during  data-collection  tasks. 

The  following  organizations  were  selected  for  interviews,  based  on  the  selection  criteria,  their 
desire  to  participate,  and  the  resources  available  to  conduct  the  interview: 
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Lockheed  Martin  M&DS 


•  Automatic  Data  Processing 

•  Bank  of  America  •  Northrop  Grumman  IT 

•  Electronic  Data  Systems  •  McKesson  Corporation 

•  John  Hancock  Financial  Services  •  Raytheon  Space  and  Airborne 

Candidate  organizations  were  chosen  using  the  following  selection  criteria  designed  to  ensure  that 
a  cross-section  of  organization  types  and  issues  were  examined: 

1 .  At  least  50%  of  the  organizations  to  be  interviewed  must  be  IT,  IS,  or  commercial  software 
development  and  maintenance  organizations  (could  be  a  sub-organization  of  a  non-IT  or¬ 
ganization). 

2.  The  remainder  of  organizations  should  represent  a  variety  of  sectors  (e.g.,  defense  contrac¬ 
tors,  automotive,  telecommunications,  banking,  financial,  and  insurance). 

3.  At  least  one  organization  must  be  a  legacy  SW-CMM  user. 

4.  A  variety  of  maturity  levels  (2  through  5)  against  SW-CMM  or  CMMI  should  be  represented 
(i.e.,  at  least  one  organization  at  each  level). 

5.  A  variety  of  organization  sizes  should  be  represented. 

6.  Participants  should  represent  issues  in  one  or  more  of  the  following  categories: 
interpretation  of  phrases  and  concepts,  generic  goals  and  generic  practices,  process  areas, 
CMMI  transition. 

7.  Participants  should  reflect  both  recent  adopters  of  CMMI  and  those  who  adopted  CMMI 
years  ago. 

8.  Include  at  least  one  organization  that  has  chosen  not  to  adopt  CMMI. 

9.  Include  at  least  one  organization  that  has  adopted  CMMI  but  is  otherwise  new  to  CMMs. 

10.  Consider  organizations  that  have  submitted  more  than  10  questionnaire  responses. 

1 1 .  Include  at  least  one  international  company. 

12.  Include  at  least  one  organization  that  is  using  CMMI  with  other  standards/approaches  (e.g., 
Six  Sigma,  ISO  9001:2000). 

13.  Include  no  more  than  one  organization  from  the  same  company/enterprise. 

14.  Consider  organizations  highly  recommended  by  the  Interpretive  Guidance  Expert  Group  or 
an  Interpretive  Guidance  Project  Team  member. 

15.  Consider  organizations  that  have  developed  interpretive  guidance  (or  similar  artifacts)  that 
the  organization  would  be  willing  to  share. 

16.  Consider  well-regarded  industry  leaders. 

The  Interpretive  Guidance  Project  Team  developed  an  initial  set  of  generalized  questions  de¬ 
signed  to  elicit  more  information  about  CMMI  within  the  targeted  organization.  These  questions 
focused  around  the  following  general  topics: 

•  what  in  CMMI  works  for  the  organization 

•  what  in  CMMI  does  not  work  for  the  organization 
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•  obstacles  encountered  by  the  organization 

•  how  the  organization  dealt  with  obstacles 

•  examples  of  what  the  organization  has  done  in  their  adoption  of  CMMI  (e.g.,  templates,  in¬ 
terpretation  notes,  policy  guidelines,  procedure  notes,  and  Paining  materials) 

In  the  next  section,  the  team’s  analysis  of  the  data  collected  is  described.  Included  in  this  section 
are  the  number  and  types  of  comments  received  and  a  characterization  of  these  comments. 
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4  Data  Analysis 


In  an  effort  to  reach  a  broad  sampling  of  the  software,  IT,  and  IS  communities,  data  collection 
sessions  were  held  at  conferences,  SPIN  meetings,  and  training  sessions;  a  Web-based  question¬ 
naire  was  administered;  and  detailed  interviews  were  conducted  with  candidate  organizations. 

Appendix  A  contains  a  summary  of  responses  to  general  questions  from  the  Interpretive  Guidance 
Questionnaire.  This  summary  provides  a  sampling  of  opinions  about  CMMI.  More  information 
about  the  responses  to  questions  from  the  Interpretive  Guidance  Questionnaire  is  available  in  the 
report  CMMI  Interpretive  Guidance  Project:  Preliminary  Report,  a  companion  report  to  this  one 
[Chrissis  2003a], 

The  Interpretive  Guidance  Project  team  collected  about  7,500  comments  on  specific  issues  from 
the  BOF  sessions,  the  online  survey,  and  the  detailed  interviews  and  compiled  them  into  a  data¬ 
base5.  This  section  discusses  the  analysis  of  these  comments. 

The  analysis  process  was  designed  to  help  the  team  determine  (1)  if  inteipretive  guidance  was 
needed,  (2)  where  interpretive  guidance  was  appropriate,  (3)  what  form  interpretive  guidance 
should  take,  and  (4)  whether  there  were  other  activities  outside  of  interpretive  guidance  that 
would  address  the  comments  received.  To  help  make  this  determination,  the  comments  were  first 
categorized  into  the  following  groups: 


1. 

CMMI  Models 

8. 

Benefits  Evidence  and  Case  Studies 

2. 

Using  CMMI  with  Other  Standards 

9. 

CMMI  Training  Courses 

3. 

CMMI  Model  Coverage 

10. 

General  Interpretation 

4. 

Cost  of  CMMI  Adoption 

11. 

CMMI  for  Small  Organizations 

5. 

Process  Improvement  Guidance 

12. 

How  to  Get  Started 

6. 

SCAMPI  Appraisals 

13. 

CMMI  Marketing  and  Communication 

7. 

CMMI  Model  Representations 

14. 

High  Maturity  Guidance 

Figure  1:  The  Categories  Used  to  Characterize  Comments 


5  For  the  purpose  of  this  report,  only  the  comments  on  specific  issues  that  were  collected  are  discussed. 


CMU/SEI-2004-SR-008 


11 


Within  each  of  these  categories,  there  were  multiple  comments  that  ranged  from  those  relating  to 
interpretation  of  the  model  to  those  that  suggested  that  there  could  be  errors  or  improvements  that 
could  be  made  to  the  product  suite.  For  example,  within  the  category  of  CMMI Models,  there 
were  comments  that  discussed  a  particular  aspect  of  the  Causal  Analysis  and  Resolution  process 
area  that  caused  difficulty  for  an  organization  during  adoption  as  well  as  comments  that  specifi¬ 
cally  asked  how  a  particular  generic  practice  relates  to  a  process  area.  There  were  also  comments 
suggesting  that  process  areas  be  combined  or  deleted. 


As  illustrated  in  Figure  2,  the  largest  number  of  comments  covered  topics  specific  to  CMMI 
models.  There  were  3,458  comments,  or  over  45  percent  of  all  comments  received,  that  pertained 
to  confusing  content  or  defects  found  in  CMMI  models.  Other  topics  that  got  a  lot  of  attention 
included  the  following: 


•  Expanding  coverage  of  CMMI  models  to  new  areas 

•  Using  CMMI  with  other  standards 

•  Reducing  the  cost  of  adopting  CMMI 


Figure  2:  Categories  of  Comments  and  Their  Relative  Number 
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After  organizing  the  comments  into  these  14  categories,  Interpretive  Guidance  Project  Team 
members  read  each  comment  and  determined  whether  the  comment  was  positive,  negative,  or 
non-actionable  and  also  whether  a  comment  should  be  categorized  as  open  or  closed.  All  positive 
and  non-actionable  comments  were  categorized  as  “closed”  because  no  further  processing  of  the 
comment  was  required.  There  were  about  4,800  comments  closed  by  the  team. 


About  28  percent  of  the  closed  comments  provided  positive  feedback.  Some  examples  of  positive 
comments  received  are  listed  below: 

•  CMM1  effectively]  establishes  an  understandable  framework,  in  which  an  organization 
can  ferret  out  problem  areas,  and  develop  appropriate  improvements. 

•  CMMI  practices  help  eliminate  loss  of  information  about  a  process  over  time,  allowing 
for  more  reuse  of  legacy  development. 

•  I  have  been  involved  with  two  different  types  of  organizations  on  their  process  improve¬ 
ment  efforts  at  an  EPG  or  program  management  level  and  I  feel  it  is  a  great  model  for 
guiding  process  improvement  for  any  organization  whether  large  or  small.  It  is  easy  to  in¬ 
terpret  and  to  me  is  common  sense. 

•  There  appears  to  be  adequate  classes,  seminars,  and  other  organizations  that  serve  as  re¬ 
sources  to  gain  answers  to  questions  in  implementing  CMMI. 

Where  appropriate,  the  positive  comments  were  included  in  change  requests  so  a  balanced  picture 
of  the  issue  was  presented  whenever  both  positive  and  negative  feedback  on  the  same  issue  was 
received.  The  comments  that  were  purely  positive  and  not  linked  to  any  other  issues  were  closed 
and  no  further  processing  was  done. 


About  72  percent  of  the  closed  comments  were  non-actionable.  Some  examples  of  non-actionable 
comments  were  comments  such  as  “ not  applicable ,”  “none,”  and  “I  have  not  yet  spent  enough 
time  assessing  this  to  provide  a  meaningful  answer.”  Although  these  comments  provided  answers 
to  the  survey  questions,  there  was  no  further  processing  necessary. 
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Of  the  approximate  7,500  comments,  64  percent  were  closed,  leaving  about  2,700  open. 


Figure  3:  Percentage  of  Comments  Open  and  Closed 

These  2700  open  comments  were  subject  to  further  analyses.  The  results  of  these  analyses  are 
described  in  the  next  section. 
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5  Analysis  Results 


Once  the  open  comments  in  the  Interpretive  Guidance  database  were  analyzed,  many  were  con¬ 
solidated  into  change  requests  and  submitted  as  part  of  the  CMMI  User  Feedback  Process6.  These 
change  requests  will  be  analyzed  and  addressed  as  paid  of  the  CMMI  Version  1.2  development 
process,  which  is  expected  to  result  in  an  updated  CMMI  Product  Suite  in  2006. 


There  were  312  change  requests  created  as  a  result  of  analyzing  approximately  2,200  comments, 
which  reflect  81  percent  of  the  open  comments.  Each  change  request  covered  anywhere  from  1  to 
30  comments.  Of  these  change  requests,  288  applied  to  model-related  issues,  15  applied  to  ap¬ 
praisal-related  issues,  and  9  applied  to  training-related  issues.  These  change  requests  will  be  used 
to  guide  changes  to  CMMI  products.  The  comments  that  are  addressed  by  change  requests  will 
not  be  used  for  developing  interpretive  guidance. 


Change  Requests  Submitted 


Training 


Figure  4:  Type  and  Number  of  Change  Requests  Submitted 

There  were  527  comments,  roughly  20  percent,  that  pertained  to  understanding  and  interpreting 
the  CMMI  Product  Suite.  These  comments  form  the  basis  for  creating  interpretive  guidance  mate¬ 
rials,  which  will  include  papers,  frequently  asked  questions  (FAQs),  new  course  materials,  and 
CMMI  adoption  aids. 


For  more  information  about  the  CMMI  User  Feedback  Process,  refer  to 
http://www.sei.cmu.edu/cmmi/models/change-requests.html. 


CMU/SEI-2004-SR-008 


15 


Figure  5  summarizes  the  fourteen  categories  identified  in  Figure  1 . 


Number  of  Comments  by  Category 

CMMI  Models 
Using  CMMI  with  Other  Standards 
CMMI  Model  Coverage 
Cost  of  CMMI  Adoption 
Process  Improvement  Guidance 
SCAMPI  Appraisals 
CMMI  Model  Representations 
Benefits  Evidence  and  Case  Studies 
CMMI  Training  Courses 
General  Interpretation 
CMMI  for  Small  Organizations 
How  to  Get  Started 
CMMI  Marketing  and  Communication 
High  Maturity  Guidance 

0  500  1000  1500  2000  2500  3000  3500  4000 

Figure  5:  Number  of  Interpretive  Guidance  Comments  by  Category 

The  rest  of  this  section  contains  category  summaries  presented  in  order  by  the  number  of  com¬ 
ments  submitted  (most  to  least).  Each  section  contains  a  description  of  the  comments  received, 
the  number  of  comments  received,  some  sample  comments,  and  SEI  activities  that  address  the 
issues  identified. 


5.1  CMMI  Models 

There  were  3,458  comments  submitted  (2,073  closed  and  1,385  open)  that  addressed  the  content 
or  structure  of  the  CMMI  models. 


CMMI  Models 


0  500  1000  1500  2000 

Figure  6:  Comments  About  CMMI  Models 
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These  comments  identified  confusing  areas  or  defects  in  CMMI  models  and  ranged  from  requests 
to  provide  help  in  applying  generic  practice  2.2  to  suggestions  for  combining  two  specific  prac¬ 
tices  into  one  in  the  Project  Monitoring  and  Control  process  area.  There  were  comments  that 
called  for  improving  the  practices  related  to  integrated  product  and  process  development  (IPPD) 
to  be  more  concrete  and  useful.  Most  of  these  comments  were  submitted  as  change  requests  for 
Version  1 .2  and  are  also  summarized  by  process  area  and  generic  practice  in  Appendix  B  of  this 
report. 


Below  are  a  few  representative  comments  provided  by  participants: 

•  I  like  the  organization  of  level  4  PAs/practices  by  organization  and  project,  rather  than 
between  process  and  product  as  in  SW-CMM. 

•  So  glad  that  we've  moved  away  from  subcontract  management  to  what  really  happens  ... 
supplier  relationships.  This  also  has  created  a  bridge  in  our  organization  between  the 
supplier  sourcing  experts  and  the  CMMI/process  improvement  teams. 

•  All  SPs  are  useful  to  my  organization.  Requirement  management  is  always  a  big  issue  in 
my  organization,  but  we  did  not  do  it  well  before,  after  adopting  CMMI  REQM  PA,  it's 
really  a  good  reference  for  us  to  manage  requirements  better. 

•  Common  features  are  more  for  historical  in  terest  than  of  any  practical  use. 

•  GP  2.2  is  confusing  and  needs  much  better  explanation  and  interpretation  with  practical 
examples. 

•  There  is  confusion  about  the  interpretations  of  the  relationship  between  strategic  and  tac¬ 
tical  training  needs. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Analyzing  the  change  requests  created  from  comments  received  as  part  of  this  project  for 
the  development  of  CMMI  Version  1.2. 

•  Using  comments  to  guide  the  selection  of  information  to  be  published  on  the  CMMI  Web 
site. 

•  Providing  help  in  applying  particular  specific  and  generic  practices  on  the  SEI  Web  site 
as  part  of  the  CMMI  FAQs.  To  review  the  model-related  FAQs,  see 
http://www.sei.cmu.edu/cmmi/faq/grp01-faq.html. 


5.2  Using  CMMI  with  Other  Standards 

There  were  761  comments  were  received  regarding  using  CMMI  with  other  standards  (718  closed 
and  43  open). 
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Figure  7:  Comments  About  Using  CMMI  with  Other  Standards 

In  today’s  market,  many  organizations  use  various  quality  standards  and  approaches  such  as  Six 
Sigma,  ISO,  and  PSP/TSP  to  meet  the  objectives  of  their  organizations.  CMMI  must  remain 
flexible  to  allow  these  organizations  to  more  easily  harmonize  their  processes  with  both  CMMI 
and  their  other  standards.  Below  are  a  few  representative  comments  provided  by  participants: 

•  We're  hoping  that  the  CMMI  will  reduce  the  division-specific  process  improvement  efforts 
encouraged  by  the  various  CMMs  through  the  integration  of  the  models. 

•  Other  models  and  standards  may  be  more  suitable  for  specific  applications,  but  not  for 
overall  organizational  process  improvement. 

•  The  CMMI  in  tegration  of  systems  and  sendees  with  software  has  provided  the  impetus  for 
us  to  apply  formed  processes  in  a  wider  variety  of  projects.  The  integration  has  made  this 
easier  since  my  part  of  the  organization  is  not  a  purely  engineering  shop,  more  of  a  sys¬ 
tems  integrator.  The  very  much  improved  Supplier  Sourcing  has  also  provided  a  more 
pragmatic  approach  to  subcontract  management. 

•  We  do  need  to  better  understand  and  benefit  from  agile  technologies. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Adding  information  about  using  CMMI  with  other  standards  or  approaches,  including 
comparisons  and  mappings  to  other  process-related  standards,  to  the  SEI  Web  site  as  it 
becomes  available. 

•  Providing  a  number  of  presentations  and  reports  on  the  SEI  Web  site  at 
http://www.sei.cmu.edu/cmmi/adoption/how-to.html.  Most  of  this  information  is  in  the 
form  of  presentations  created  for  conferences  such  as  the  Software  Engineering  Process 
Group  (SEPG)  conference  and  the  CMMI  Technology  Conference  and  User  Group. 

•  Increasing  SEI  staff  participation  in  the  IEEE  and  International  Organization  for  Stan¬ 
dardization  (ISO)  standards  communities. 

•  Selecting  information  to  add  to  the  CMMI  Web  site  based,  in  part,  on  the  comments  re¬ 
ceived. 

•  Planning  pilot  SCAMPI  appraisals  that  produce  dual-outcome  appraisal  results. 
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5.3  CMMI  Model  Coverage 


There  were  726  comments  received  (667  closed  and  59  open)  that  suggested  that  the  coverage  of 
CMMI  models  be  expanded  to  include  additional  disciplines,  a  greater  part  of  the  product  lifecy¬ 
cle,  and  more. 


CMMI  Model  Coverage 


Open  59 

Closed  667 

0 

500 

1000 

1500 

2000 

Figure  8:  Comments  About  CMMI  Model  Coverage 

The  most  commonly  mentioned  areas  for  possible  expansion  included  acquisition,  services,  op¬ 
erations,  and  hardware.  Below  are  a  few  representative  comments  provided  by  participants: 

•  If  and  when  the  SA-CMM  is  also  included  in  CMMI  it  will  be  much  more  beneficial  to  my 
organization. 

•  Yes,  but  it  does  not  cover  all  types  of  IT  projects  or  those  that  do  not  follow  a  full  lifecy¬ 
cle. 

•  We  ( the  CMMI  community)  need  to  include  hardware  development. 

•  It’d  be  even  better  if  there  was  a  version  of  the  model  that  didn't  even  mention  engineer¬ 
ing. 

•  Broaden  or  interpret  CMMI  SW/SE  to  include  ongoing  operations  of  our  other  communi¬ 
cations  systems. 

•  We  had  hoped  the  CMMI  model  would  be  useful  to  a  non-software  development,  diversely 
operational  organizational  such  as  ours,  but  we  found  it  to  be  too  granular,  too  subjec¬ 
tive,  and  too  project-centric. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Analyzing  the  change  requests  created  from  comments  received  as  part  of  this  project  for 
the  development  of  CMMI  Version  1.2. 

•  Considering  appropriate  expansion  areas  for  CMMI  Version  1.2  and  beyond.  (CMMI 
Steering  Group  approval  is  required  for  CMMI  model  expansion.) 
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5.4  Cost  of  CMMI  Adoption 

The  team  received  613  comments  regarding  CMMI  adoption  costs:  451  closed  and  162  open. 


Cost  of  CMMI  Adoption 


Open  162 


Closed  451 


0  500  1000  1500  2000 


Figure  9:  Comments  About  the  Cost  of  CMMI  Adoption 

The  number  of  positive  comments  (e.g.,  those  that  believed  that  the  value  of  CMMI  process  im¬ 
provement  outweighed  appraisal  and  training  costs)  and  the  number  of  negative  comments  were 
nearly  balanced.  Below  are  a  few  representative  comments  provided  by  participants: 

•  “Considering  the  cost  of  not  adopting  the  CMMI,  the  cost  of  adoption  is  not  that  great.  ” 

•  The  cost  of  adopting  CMMI  is  truly  minimal  if  an  organization  embraces  CMMI  fully.  It's 
only  when  CMMI  is  an  "added”  requirement  that  it  is  costly. 

•  I  believe  the  clear,  specific,  and  relev ant-to-us  structure  and  guidance  provided  by  CMMI  will 
reduce  cost  of  using  it  compared  to  less  applicable  models  like  ISO. 

•  The  cost  of  the  appraised  and  evidence  collection  is  staggering.  Because  we  are  reaching  into 
more  organizations  than  before,  and  reaching  deeper  into  their  process,  there  is  definitely 
more  cost. 

•  It’s  not  the  cost  of  Process  Improvement — that  pays  for  itself— it’s  the  cost  of  appraisals  that 
slow  us  down. 

•  Although  courses  are  available,  they  are  cost  prohibitive  (a  deterrent )  to  the  whole  effort. 

These  comments  are  difficult  to  satisfactorily  address  because  some  supported  while  others  dis¬ 
puted  the  existence  of  a  problem  with  CMMI  adoption  costs.  Further,  balancing  costs  with  the 
quality  of  products  and  services  must  be  considered. 


How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 
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•  Using  comments  about  the  cost  of  appraisals,  course  attendance,  and  licensing  as  inputs 
to  decision  making  related  to  the  SEI  Partner  Network,  SEI  Education  and  Training,  and 
the  CMMI  Initiative. 

•  Gathering  CMMI  cost  and  benefits  information  that  will  be  periodically  reported  to  the 
community.7  Organizations  are  invited  to  participate  in  this  effort  to  document  results  in 
multiple  ways.  The  SEI  is  also  coordinating  the  creation  of  tracks  at  multiple  conferences 
where  organizations  present  their  CMMI  data. 

•  Providing  on  the  SEI  Web  site  quantitative  information  regarding  adoption  costs  and 
benefits  as  reported  in  presentations  and  papers  at 
http://www.sei.cmu.edu/cmmi/adoption/concepts.html. 

•  Providing  other  resources,  such  as  the  SEI’s  Software  Engineering  Information  Reposi¬ 
tory  (http://seir.sei.cmu.edu),  the  CMMI  Technology  Conference  and  User  Group 
(http://www.sei.cmu.edu/cmmi/events/cmmi-techconf.html),  and  the  Software  Engineer¬ 
ing  Process  Group  Conference  (http://www.sei.cmu.edu/sepg/)  where  users  can  interact 
with  other  CMMI  adopters  and  find  information  and  presentations  regarding  CMMI 
adoption  costs  and  benefits. 


5.5  Process  Improvement  Guidance 

Four  hundred  twenty-six  comments  (333  closed  and  93  open)  addressed  process  improvement 
guidance. 


Process  Improvement  Guidance 


Open  93 


Closed  333 
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Figure  1 0:  Comments  About  Process  Improvement  Guidance 

Many  respondents  felt  that  the  CMMI  Product  Suite  and  its  supporting  materials  were  adequate 
for  providing  process  improvement  guidance.  Below  are  a  few  representative  comments  provided 
by  participants: 

•  It  is  a  lot  more  descriptive  than  the  SW-CMM,  and  seems  to  be  updated  with  more  knowl¬ 
edge  on  the  curren  t  state  of  software  developmen  t. 


In  2003,  the  first  report  addressing  CMMI  costs  and  benefits.  Demonstrating  the  Impact  and  Benefits  of  CMMI: 
An  Update  and  Preliminary  Results ,  became  available  on  the  SEI  Web  site  at 
http://www.sei.cmu.edu/publications/documents/  03.reports/03sr009.html  [Goldenson  2003]. 
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•  CMMI  is  a  good  reference  ‘architecture’ for  process  improvement.  It  also  is  a  good  ‘me¬ 
dia  ’for  communicating  between  engineering  side  and  management  side. 

•  Six  Sigma  has  proven  to  be  a  good  mechanism  for  process  improvemen  ts.  This  is  com¬ 
patible  with  the  CMMI,  not  a  different  approach. 

•  One  problem  is  that  process  improvemen  t  is  viewed  as  an  overhead  activity. 

•  The  model  doesn  't  solve  everything,  so  our  organization  still  needs  a  general  process  im¬ 
provement  model.  We  use  an  adaptation  of  Six  Sigma  and  the  process  improvemen  t  circle. 

However,  other  respondents  said  that  they  needed  help  with  process  improvement  tasks.  This  type 
of  information  is  useful  to  any  process  improvement  program,  not  just  those  programs  using 
CMMI  for  their  process  improvement. 


How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Using  comments  indicating  the  need  for  additional  process  improvement  guidance  to 
guide  the  development  of  new  products  and  services  to  help  those  conducting  process 
improvement  in  their  organizations. 

•  Using  comments  to  help  determine  which  questions  to  add  to  the  CMMI  Frequently 
Asked  Questions,  published  on  the  SEI  Web  site  at 
http://www.sei.cmu.edu/cmmi/adoption/ques-ans.html. 

•  Supporting  a  Partner  Network  that  assists  organizations  with  their  process  improvement 
needs.  More  information  about  the  SEI  Partner  Network  is  available  on  the  SEI  Web  site 
at  http://www.sei.cmu.edu/collaborating/partners/trans.partners.html. 

•  Offering  several  hands-on  SEI  courses  to  help  those  building  a  new  process  improvement 
program  or  operating  an  existing  one.  These  process  improvement  related  courses  are 
listed  on  the  SEI  Web  site  at 

http://www.sei.cmu.edU/products/courses/courses.html#PROC. 

•  Offering  certificate  programs  that  organize  SEI  courses  into  a  meaningful  sequence  of 
learning  about  broader  process  improvement.  These  certificate  programs  are  outlined  on 
the  SEI  Web  site  at  http://www.sei.cmu.edu/activities/credentials/programs.html. 

•  Planning  a  CMMI  interpretation/implementation  course  and  using  comments  indicating 
the  need  for  additional  process  improvement  guidance  to  help  define  the  requirements  for 
this  course. 
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5.6  SCAMPI  Appraisals 

The  project  received  391  comments  regarding  SCAMPI  Appraisals:  246  closed  and  145  open. 


SCAMPI  Appraisals 


Open  145 


Closed  246 
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Figure  1 1 :  Comments  About  SCAMPI  Appraisals 

These  comments  addressed  specific  experiences  with  SCAMPI  Class  A  appraisals.  The  most  fre¬ 
quent  comments  involved  the  need  to  conduct  appraisals  using  a  less  formalized  approach.  Other 
comments  included  concerns  about  how  to  apply  particular  specific  or  generic  practices  to  situa¬ 
tions  in  which  they  did  not  seem  to  apply  or  seemed  to  overlap  with  other  CMMI  process  areas. 
Also  received  were  requests  for  clarification  of  particular  parts  of  the  appraisal  process  and  re¬ 
quests  for  additional  supporting  materials.  The  team  also  received  positive  comments.  Below  are 
a  few  representative  comments  provided  by  participants: 

•  The  methods  are  suitable.  Having  Class  B  and  C  appraisals  function  as  health  checks, 
which  we  rely  on  as  a  way  to  gauge  progress  without  the  added  burden  of  trying  to 
achieve  a  rating. 

•  Really  like  the  ability  to  give  project-specific  findings 

•  The  CMMI  appraisal  methods  provide  good  guidance  to  help  organisations  to  find  op¬ 
portunities  to  improve. 

•  SCAMPI  method  is  expensive,  hard  to  implement. 

•  More  guidance  is  needed  for  conduct  of  SCAMPI  B  and  C  assessments. 

•  Believe  SCAMPI  is  going  to  be  challenging  without  automated  tool  support  as  amount  of 
data  and  results  to  be  processed  and  collated  increased  greatly. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Analyzing  the  change  requests  created  from  comments  received  as  part  of  this  project  for 
the  development  of  CMMI  Version  1.2. 8 


Part  of  Version  1.2  will  be  updating  the  Standard  CMMI  Appraisal  Method  for  Process  Improvement  (SCAMPI) 
Method  Definition  Document  (MDD)  and  Appraisal  Requirements  for  CMMI  (ARC)  documents. 
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•  Developing  and  piloting  SCAMPI  Class  B  and  C  appraisals  (available  for  public  use  in 
November  2004)  guided,  in  part,  by  the  comments  received.9 

•  Providing  frequently  asked  questions  (FAQs)  on  the  SEI  Web  site  that  answer  many  of 
the  questions  about  SCAMPI  appraisals  These  FAQs  are  found  at 
http://www.sei.cmu.edu/cmmi/faq/gip02-faq.html. 

•  Providing  materials  from  delivered  presentations  that  describe  the  experiences  of  differ¬ 
ent  organizations  that  have  conducted  appraisals  on  the  SEI  Web  site  at 
http://www.sei.cmu.edU/cmmi/appraisals/#getting-started. 

•  Supporting  SEI  Partners  which  conduct  SCAMPI  appraisals.  These  Partners  are  listed  on 
the  SEI  Web  site  at 

http://www.sei.cmu.edU/collaborating/partners/partners-tech.html#SCAMPI. 


5.7  CMMI  Model  Representations 

There  were  316  comments  received  that  covered  the  topic  of  model  representations  (76  closed 
and  240  open). 


CMMI  Model  Representations 
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Figure  12:  Comments  About  CMMI  Model  Representations 

Many  respondents  expressed  their  desire  to  have  only  one  model  representation  instead  of  two. 
The  problem  was  that  there  was  no  agreement  on  which  representation  to  remove  and  which  to 
keep.  There  were  close  to  fifty  comments  received  in  which  the  respondent  wanted  only  the  Con¬ 
tinuous  representation.  Likewise,  there  were  about  fifty  comments  where  the  respondent  wanted 
only  the  Staged  representation.  And  there  were  about  fifty  comments  where  respondents  didn’t 
care  which  representation  the  CMMI  Project  wanted  to  adopt  as  long  as  there  was  only  one  repre¬ 
sentation  supported. 


There  were  90  other  comments  that  wanted  support  for  both  representations  to  continue  but  also 
wanted  the  differences  between  the  representations  to  be  explained  better.  Plus,  the  SEI’s  adop- 


With  the  release  of  these  appraisal  methods,  many  of  the  comments  received  will  be  addressed.  CMMI  also  allows 
an  organization  to  develop  and  use  other  Class  B  or  C  appraisal  methods  that  meet  the  Appraisal  Requirements  for 
CMMI  [ARC], 
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tion  statistics  show  significant  use  of  both  representations.  Below  are  a  few  representative  com¬ 
ments  provided  by  participants: 

•  It  is  easier  to  train  from  the  con  tin  uous  view,  but  it  is  easier  to  implemen  t  from  the  staged 
view. 

•  We  use  the  staged  representation  but  our  customers  use  ISO  15504.  So  mapping/switching  is 
supported  by  CMMI. 

•  Even  though  we  have  not  begun  CMMI,  I  believe  that  having  a  choice  is  very  beneficial  for 
organizations. 

•  The  staged  vs.  continuous  representations  are  difficult  for  some  to  understand. 

•  The  staged  representation  is  a  moderate  improvement  on  the  SW-CMM.  The  continuous  rep¬ 
resentation  is  more  problematic  and  opens  up  issues  that  the  SW-CMM  did  not  have. 

•  The  continuous  representation  is  the  approach  we're  using  for  the  moment.  But,  since  we 
can  't  get  a  rating  based  on  it,  I  have  no  doubt  that  external  and  management  pressures  will 
force  us  to  move  to  the  staged  approach.  Not  sure  what  the  real  impact  to  the  organization  's 
process  improvemen  t  efforts  will  be  as  a  result  of  our  decision. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Analyzing  the  change  requests  created  from  comments  received  as  part  of  this  project  for 
the  development  of  CMMI  Version  1.2. 

•  A  course  that  combines  the  concepts  from  the  Staged  and  Continuous  introductory 
courses  into  a  single  course,  Introduction  to  CMMI  (Staged  and  Continuous),  is  currently 
under  development.  Part  of  Version  1.2  development  will  be  updating  all  CMMI-related 
courses,  including  this  new  one. 

•  Explaining  and  comparing  CMMI  model  representations  in  a  number  of  places,  including 
the  CMMI  book  published  by  Addison-Wesley,  CMMI:  Guidelines  for  Process  Integra¬ 
tion  and  Product  Improvement,  the  CMMI  Web  site,  and  articles  such  as  Choosing  a 
CMMI  Model  Representation,  found  on  the  Web  at 

http://www.stsc.hill.af.mil/crosstalk/2000/07/shrum.html  [Chrissis  2003b,  Shrum  2000]. 

•  Continuing  to  support  both  representations,  but  packaging  the  model  in  Version  1.2  to  be 
in  a  format  similar  to  the  format  used  in  the  CMMI  book,  CMMI:  Guidelines  for  Process 
In  tegration  and  Product  Improvemen  t. 

5.8  Benefits  Evidence  and  Case  Studies 

There  were  270  comments  (64  closed  and  206  open)  were  received  that  were  related  to  CMMI 

benefits  evidence. 
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Benefits  Evidence  and  Case 
Studies 


Open  206 


Closed  64 
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Figure  13:  Comments  About  CM  Ml  Benefits  Evidence  and  Case  Studies 

Many  of  the  comments  described  the  need  for  evidence  of  CMMI  benefits,  ROI  evidence  in  par¬ 
ticular,  to  help  them  convince  their  executives  and  other  influential  decision  makers  that  CMMI  is 
a  wise  choice  for  process  improvement.  Many  such  respondents  suggested  that  CMMI  adoption 
would  be  easier  if  such  evidence  were  up  to  date  and  easily  available.  Below  are  a  few  representa¬ 
tive  comments  provided  by  participants: 

•  We  have  been  doing  so  for  the  past  year  and  notice  that  the  organization  is  making  good 
returns  on  the  investment.  Plus  better  understanding  of  the  CMMI  Model  seems  to  help 
the  projects  improve. 

•  It  helps  but  it  is  not  absolutely  essen  tial.  The  marketing  aspect  of  being  able  to  say  that 
one  is  at  level  3,  4  or  5  also  carries  some  weight. 

•  We  know  we  are  getting  value  out  of  it  and  we  are  doing  some  measures  to  prove  it.  If 
there  were  other  measures ,  it  would  help,  but  it  won  't  drive  us  to  do  anything  differently. 

•  Absence  of  ROI  evidence  is  in  many  cases  a  barrier  when  discussing  the  CMMI  with  in¬ 
ternal  organizations. 

•  Adopting  CMMI  costs  tremendous  amount  money.  Any  method  to  measure  cost  effective 
analysis  will  be  a  key  aspect  for  continuing  secure  sponsorship  from  upper  management. 

•  I  have  trouble  convincing  senior  management  of  the  benefits  they  will  receive  in  imple¬ 
menting  CMMI.  This  results  in  a  lack  of  support  from  them. 

Case  studies  were  mentioned  time  and  again  as  an  effective  way  to  communicate  benefits  evi¬ 
dence.  However,  there  were  respondents  that  felt  that  external  ROI  data  was  unnecessary  and  they 
were  using  their  own  data  to  justify  the  need  for  and  value  of  CMMI. 


How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Collecting  CMMI  benefits  evidence  and  using  the  comments  received  to  help  better  un¬ 
derstand  the  needs  of  the  community.  A  preliminary  report  publishing  benefit  evidence 
was  released  in  late  2003  after  the  Interpretive  Guidance  on-line  survey  was  conducted. 
This  report.  Demonstrating  the  Impact  and  Benefits  of  CMMI:  An  Update  and  Prelimi- 
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nary  Results,  is  available  on  the  SEI  Web  site  at 

http ://ww w. sei. emu . edu/public ations/documents/03  .reports/03 sr009 . html  [Goldenson 
2003], 

•  Conducting  ongoing  data  collection  and  analysis  and  collaborating  with  CMMI  adopters 
to  facilitate  the  collection  of  additional  empirical  evidence  on  the  cost  and  benefits  of 
adopting  CMMI.  As  this  information  is  collected,  verified,  and  analyzed,  it  will  provide 
additional  evidence  to  the  public  in  different  forms,  including  case  studies,  tutorials,  ROI 
workshops,  summary  presentations,  and  reports.  A  comprehensive  technical  report  is 
planned  to  be  released  in  September  2005. 


5.9  CMMI  Training  Courses 

There  were  267  (153  closed  and  114  open)  comments  received  asking  for  specific  improvements 
to  CMMI-related  courses. 


CMMI  Training  Courses 

Open  114 
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Figure  14:  Comments  About  CMMI  Training  Courses 

A  majority  of  these  comments  asked  that  more  of  an  implementation  focus  be  available  as  part  of 
CMMI  training.  Some  of  the  comments  contained  specific  suggestions  for  improving  the  existing 
CMMI  introductory  courses.  These  comments  were  converted  to  change  requests  and  submitted 
to  the  team  responsible  for  CMMI  Version  1.2  revision  efforts.  Many  other  comments  contained 
concerns  about  training  costs.  These  comments  were  provided  to  the  SEI  Education  and  Training 
group.  Below  are  a  few  representative  comments  provided  by  participants: 

•  There  appears  to  be  adequate  classes,  seminars,  and  other  organizations  that  sen’e  as  re¬ 
sources  to  gain  answers  to  questions  in  implementing  CMMI. 

•  Existing  CMMI  Training  was  excellent.  Our  organization  requirements  are  not  just  soft¬ 
ware.  More  is  needed. 

•  Relevant  technical  training  should  be  more  emphasized  than  model  training,  e.g.  process 
improvemen  t,  process  definition,  although  SEI  provides  those,  people  tend  to  focus  on 
model  first.  SEI  should  mentor  all  those  who  just  start  SPI. 

•  The  CMMI  training  is  still  mostly  abstract.  People  need  very  focused  training  to  under¬ 
stand  what  it  means  to  their  organization. 
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•  Our  organization  (and  I  believe  countless  others)  would  benefit  from  CMMI  training  that 
ties  roles  (e.g.,  CM,  QA,  project  manager)  to  process  areas,  typical  artifacts  and  how  it 
might  translate  into  actions  or  behaviors. 

•  Unless  something  has  changed  over  the  last  year,  I  feel  that  the  training  courses  do  not 
provide  enough  guidance  on  how  to  use,  implement,  interpret,  assess,  etc.  the  model. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Analyzing  the  change  requests  created  from  comments  received  as  part  of  this  project  for 
the  development  of  CMMI  Version  1.2. 

•  A  course  that  combines  the  concepts  from  the  Staged  and  Continuous  introductory 
courses  into  a  single  course,  Introduction  to  CMMI  (Staged  and  Continuous),  is  currently 
under  development.  Part  of  Version  1.2  development  will  be  updating  all  CMMI-related 
courses,  including  this  new  one. 

•  Planning  a  CMMI  interpretation/implementation  course.  Comments  will  be  used  to  help 
define  the  requirements  for  this  course.  This  course  will  concentrate  on  providing  inter¬ 
pretation  and  implementation  guidance  that  is  often  asked  for  by  CMMI  adopters.  It  will 
include  more  concrete  examples  and  guidance  to  help  organizations  begin  their  adoption 
of  CMMI. 

•  Using  comments  to  develop  other  new  CMMI  courseware. 


5.10  General  Interpretation 

The  team  received  175  comments  (11  closed  and  164  open)  in  which  respondents  asked  for  addi¬ 
tional  information  that  could  help  them  to  interpret  CMMI. 


General  Interpretation 
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Figure  15:  Comments  About  General  Interpretation  of  CMMI 

These  comments  raise  topics  ranging  from  how  a  particular  generic  practice  relates  to  a  process 
area  to  providing  a  consistent  list  of  roles  and  activities  for  each  process  area.  Below  are  a  few 
representative  comments  provided  by  participants: 
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•  Providing  organizations  two  representations  with  tailoring  options  for  SW,  SE,  IPPD,  and 
SS  makes  developing  a  system  approach  to  process  improvement  easier  for  large,  some¬ 
what  diverse,  organizations. 

•  As  a  Center  we  have  selected  a  specific  variation  that  we  feel  will  have  the  most  ROI  for 
our  end-users,  but  the  choice  between  two  representations  has  been  of  considerable  bene¬ 
fit.  We're  finding  that  most  projects  prefer  to  implement  the  staged  representation  in  Level 
2,  as  they  consider  that  to  implement  the  basics  of  a  good  management  infrastructure.  An 
emerging  trend  then  shows  them  to  be  switching  to  continuous  representation  in  Level  3, 
where  they  can  focus  on  those  elements  that  would  provide  the  most  immediate  ROI  and 
benefit  to  the  end-users  and  sponsors. 

•  The  value-added/or  reason  why  to  use  a  certain  type  of  model  may  need  a  little  more 
clarification  and  why  an  organization  should  use  one  or  another. 

•  It  would  be  nice  to  have  a  cross  reference  that  identifies  every  time  a  given  role  has  a  re¬ 
sponsibility.  For  example,  QA  has  a  role  in  most  process  areas.  It  would  be  helpful  to 
have  a  list  for  comparison  of  what  we've  already  have  documented,  against  what  the 
CMMI  says  QA  should  be  doing  without  having  to  search  through  every  page. 

•  Just  like  with  any  model,  people  forget  that  it  is  a  model  and  they  strive  to  implement  eve¬ 
rything  con  tained  in  the  model  and  exactly  as  described  in  the  model.  This  tends  to  focus 
on  things  other  than  process  improvement.  And  since  the  CMMI  is  more  complex  than 
other  models  it  is  easy  for  people  to  lose  focus. 

•  CMMI  provides  a  general  set  of  best  practices  within  the  scope  of  its  discipline  coverage, 
but  is  not  adequate  in  defining  how  it  may  be  applied  for  use  in  specific  situations. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Analyzing  the  change  requests  created  from  comments  received  as  part  of  this  project  for 
the  development  of  CMMI  Version  1.2,  including  CMMI  models  and  CMMI  training 
courses. 

•  Continually  updating  the  CMMI  Web  site  with  information  to  help  organizations  adopt, 
use,  and  interpret  CMMI.  For  more  information,  see  http://www.sei.cmu.edu/cmmi. 

•  Planning  a  CMMI  interpretation/implementation  course.  Comments  will  be  used  to  help 
define  the  requirements  for  this  course.  This  course  will  concentrate  on  providing  inter¬ 
pretation  and  implementation  guidance  that  is  often  asked  for  by  CMMI  adopters.  It  will 
include  more  concrete  examples  and  guidance  to  help  organizations  begin  their  adoption 
of  CMMI. 


5.11  CMMI  for  Small  Organizations 

There  were  52  comments  that  raised  the  issue  that  CMMI  does  not  adequately  meet  the  needs  of 
organizations  that  are  small  in  size.  (There  were  no  closed  comments  for  this  category.) 
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CMMI  for  Small  Organizations 


Open  52 

Closed  0 


0  500  1000  1500  2000 


Figure  1 6:  Comments  About  CMMI  for  Small  Organizations 

The  belief  is  that  CMMI  is  best  used  by  large  organizations,  and  that  additional  guidance  for 
small  organizations  would  be  helpful  in  enabling  the  translation  of  CMMI  best  practices  to  the 
environment  of  the  small  organization.  As  an  aside,  the  August  2004  CMMI  Process  Maturity 
Profile10  showed  that  38.5%  of  the  reported  SCAMPI  appraisals  were  from  organizations  with  1 
to  100  employees  and  10.4%  were  from  organizations  with  25  or  fewer  employees.  Below  are  a 
few  representative  comments  provided  by  participants: 

•  The  model  fits  large  scale  development  very  well.  However,  we  must  be  creative  when 
adapting  the  model  to  small  or  medium  sccde  projects  or  maintenance  projects.  This  can 
be  done  by  considering  alternate  practices  and  carefully  evaluating  the  risk  involved 
when  comparing  the  alternate  practice  to  the  specific  practice  in  the  model. 

•  There  have  been  numerous  process  improvement  methodologies  over  the  years.  CMMI  is 
probably  an  improvement  as  it  has  leveraged/slightly  morphed  a  number  of  them,  but 
considerable  work  needs  to  be  done  to  make  it  applicable  to  small  businesses.  In  its  cur¬ 
rent  form,  it  is  not  possible  to  apply  it  in  that  kind  of  environment.  And  guidelines  for  do¬ 
ing  so  are  sparse. 

•  Difficult  to  implement  in  small  teams.  Too  much  information  to  absorb. 

•  CMMI  is  very  detailed  down  to  the  point  of  being  nearly  procedural.  However,  it  is  diffi¬ 
cult  to  interpret  for  small  project  work  in  a  mainly  production  support  environment  espe¬ 
cially  for  the  Engineering  PAs.  Examples  and  typical  work  products  do  not  always  help  in 
this  environment. 

•  Certainly  the  idea  of  process  improvement  can  be  applied,  but  small  businesses  could  use 
a  "cookbook"  to  get  the  basics  down. 

•  Provide  more  guidance  on  how  to  adopt  the  CMMI  on  small  projects  ( 10  or  less  people) 
with  short  deadlines  (3-9  months). 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 


10  The  CMMI  Process  Maturity  Profile  is  available  at  http://www.sei.cmu.edu/sema/profile_CMMI.html. 
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•  Studying  the  use  of  CMMI  in  small  organizations  to  help  guide  their  research  and  report¬ 
ing  of  results.  The  needs  of  small  organizations  and  how  CMMI  can  be  applied  success¬ 
fully  in  these  organizations  is  being  studied.  A  presentation  describing  this  research,  is 
available  at  http://www.sei.cmu.edu/ttp/presentations/sm-bus/.  Additional  information 
can  be  found  at  http://www.sei.cmu.edu/ttp/presentations/breaking-barriers/break- 
barriers.pdf. 

•  There  are  various  presentations  available  that  address  the  issue  of  applying  CMMI  to 
small  organizations  on  the  SEI  Web  site  at 
http://www.sei.cmu.edu/cmmi/adoption/extensions.html. 

•  The  Personal  Software  Process™  (PSPSM)and  Team  Software  Process™  (TSP™)were  de¬ 
veloped  and  can,  in  addition  to  improving  teams  within  large  organizations,  directly  help 
small  organizations.  More  information  about  PSP  and  TSP  can  be  found  at 
http://www.sei.cmu.edu/tsp/. 


5.12  How  To  Get  Started 

The  team  received  34  comments  about  getting  started  with  CMMI  (1  closed  and  33  open). 


How  to  Get  Started 

Open  33 

Closed  1 
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Figure  1 7:  Comments  About  How  to  Get  Started  Adopting  CMMI 

Respondents  asked  for  guidance  in  what  to  do  first,  how  to  proceed,  and  what  to  expect.  Respon¬ 
dents  felt  that  more  guidance  was  needed  up  front  to  help  them.  Below  are  a  few  representative 
comments  provided  by  participants: 

•  The  CMMI  is  much  more  descriptive  and  complete  than  the  SW-CMM.  The  model  itself  is 
a  good  reference  for  process  improvement,  however  is  insufficient  as  the  sole  'guide'  for 
process  improvement  ( i.e.,  does  not  eliminate  the  need  for  the  IDEALSM  model). 

•  An  organization  needs  to  be  conscious  of  whether  it  is  using  the  CMMI  to  build  a  set  of 
processes  from  the  ground  up,  or  whether  it  is  using  it  to  map  an  existing  set  of  processes 
and  identify  areas  for  improvement.  If  there  is  confusion  about  the  approach  to  using  the 
model,  it  may  not  be  an  adequate  guide. 


SM  Personal  Software  Process,  PSP.  Team  Software  Process,  TSP,  and  IDEAL  are  service  marks  of  Carnegie  Mellon  Uni¬ 
versity. 
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•  Organizations  need  to  understand  that  CMMI  is  not  the  "end  all".  The  organization  's 
business  model  is  the  capstone  model  and  CMMI  (and  other  models)  are  to  support  in¬ 
creased  maturity  and  quality  in  the  business  model.  Unfortunately,  many  businesses  use 
CMMI  as  the  ad  hoc  business  model  which  (I  think)  is  incorrect.  Also,  the  word  "guiding" 
is  probably  appropriate.  The  CMMI  is  not  a  PI  process  in  itself;  most  businesses  already 
have  an  improvement  process  and  the  CMMI  provides  key  practices  /  initiatives  to  guide 
the  improvements  to  be  implemented. 

•  CMMI  in  itself  is  not  sufficient.  While  CMMI  provides  guidance  on  the  desirable  traits  to 
move  towards,  it  does  not  provide  guidance  on  how  to  implement  and  instituationalize 
change  across  an  organization.  It  tells  you  the  "what"  but  not  "how"  to  achieve  the  what. 

•  [What  is  needed  is]  a  guideline  for  those  who  are  just  beginning  to  bring  CMMI  into 
their  organizations.  Such  as  templates  for  start-up  plans,  charters,  schedules,  order  of  ac¬ 
tivities  and  time  and  people  estimates.  This  type  of  guideline  can  be  written  based  on  the 
experiences  of  those  organizations  that  have  achieved  CMMI  success. 

•  Many  cultural  and  people  issues  are  also  involved.  CMMI  is  a  good  guide,  but  it  needs  to 
be  complemented  with  methods  for  technology  transfer,  gain  ing  and  sustain  ing  organiza¬ 
tional  commitment  (and  resources),  and  a  good  marketing  plan. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Providing  a  Web  page  designed  to  help  organizations  get  started  with  CMMI  at 
http://www.sei.cmu.edu/cmmi/adoption/cmmi-start.html.  Updates  to  this  Getting  Started 
Web  page  will  be  guided,  in  part,  by  the  comments  received. 

•  Offering  the  Mastering  Process  Improvement  course,  which  provides  students  a  head  start 
in  beginning  their  process  improvement  programs.  More  information  about  this  course  is 
available  at  http://www.sei.cmu.edu/products/courses/master-process-improve.html. 

•  Designing  an  integrated  process  improvement  approach  for  incorporating  TSP  with 
CMMI,  along  with  performance  measurement,  to  accelerate  the  process  improvement 
journey  and  greatly  improve  the  results. 

5.13  CMMI  Marketing  and  Communication 

Twenty-five  comments  were  received  regarding  marketing  and  communication  (7  closed  and  1 8 

open). 
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CMMI  Marketing  and 
Communication 

Open  18 

Closed  7 
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Figure  18:  Comments  About  CMMI  Marketing  and  Communication 

The  respondents  commented  that  if  CMMI  were  more  publicized,  the  concept  would  be  known 
by  managers  and  executives.  If  decision  makers  knew  about  CMMI,  it  would  make  convincing 
them  to  adopt  CMMI  much  easier.  Some  respondents  suggested  that  the  SEI  actively  market 
CMMI  to  executives  to  increase  their  awareness  of  what  CMMI  is  and  how  it  can  help  organiza¬ 
tions  succeed.  Others  simply  wanted  CMMI  to  be  more  recognizable  by  everyone.  Below  are  a 
few  representative  comments  provided  by  participants: 

•  You  need  to  roadshow  this  thing  -  like  Rational  has  done  with  CMM / RUP.  Set  up  Webi¬ 
nars  like  Rational  does.  They  like  the  razzle-dazzle,  and  the  high-level  overviews  around 
here.  They  like  Whitepapers.  Advertise  in  Computerworld.  Offer  Whitepapers  via  Com- 
puterworld.  The  brighter  and  more  fashionable  the  "peacock",  the  more  they'll  give  it  a 
look. 

•  Promote  CMMI  as  means  not  dedicated  to  software  but  can  be  widely  used  for  any  disci¬ 
pline  in  an  organization. 

•  Greater  marketing  in  the  industry  especially  across  in  Europe.  People  still  think  this  is  an 
extremely  expensive  thing  to  do,  just  look  at  the  700  pages  and  the  size  of  evaluation 
teams.  Smaller  companies  can  use  the  model  but  just  can  't  accept  the  cost  involved  and  in 
any  case  they  can’t  see  or  don't  understand  the  advantages.  It  is  still  considered  to  be 
something  that  'Defense'  organizations  do!! 

•  Publicize.  Our  biggest  hurdle  is  that  none  of  our  board  directors  have  heard  about  it. 

They  tend  to  treat  suggestions  from  engineering  with  suspicion,  especially  where  money 
is  involved.  Perhaps  getting  articles/presentations/seminars  aimed  at  these  people  is  a 
good  step  forward. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 

•  Developing  a  short  CMMI  overview  to  specifically  target  executives  and  their  concerns. 
Comments  will  be  used  as  input  when  developing  this  new  CMMI  overview. 

•  Continually  looking  for  opportunities  to  educate  the  public  about  CMMI.  Increasingly 
there  are  articles  published  about  CMMI  in  well-known  business  publications.  Industry 
analysts  recommend  CMMI  in  their  reports.  Some  large  scale  tool  environments,  such  as 
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the  Rationale  Unified  Process  (RUP),  have  indicated  publicly  that  they  support  CMMI 
adoption. 

•  Providing  related  presentations  and  papers  on  the  SEI  Web  site  at 

http://www.sei.cmu.edu/cmmi/adoption/concepts.html.  Some  of  these  can  be  used  by 
process  improvement  champions  in  presentations  to  senior  management.  There  are  also 
increasing  numbers  of  books  available  about  CMMI.  For  more  information  about  CMMI 
books,  see  http://www.sei.cmu.edu/cmmi/adoption/books.html. 

5.14  High  Maturity  Guidance 

The  team  received  13  comments  (all  remained  open)  wanting  more  guidance  for  organizations 
pursuing  maturity  levels  4  and  5. 


High  Maturity  Guidance 

Open  13 

Closed  0 
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Figure  1 9:  Comments  About  High  Maturity  Guidance  for  CMMI 

Respondents  wanted  more  examples  of  high  maturity  practices.  Below  are  a  few  representative 

comments  provided  by  participants: 

•  In  my  experience  with  the  model,  it  seems  to  provide  good  guidance  to  build  on  to  cover 
most  situations  I  have  encountered.  I  think  Level  2  and  Level  3  offer  more  practical  ad¬ 
vice  (from  my  perspective)  than  Levels  4  and  5,  but  I  can  see  the  value  in  working  toward 
achieving  them. 

•  Because  the  CMM  frameworks  are  based  on  best  practices,  they  are  consistently  follow¬ 
ing  the  pack  in  terms  of  innovation.  Our  organization  must  constantly  be  looking  for  the 
potential  breakthroughs  in  terms  of  technology,  processes  and  resources.  We  bring  value 
to  our  customers  by  leading  the  pack,  not  just  being  a  part  of  the  pack. 

•  We  are  supplementing  CMMI  with  Six  Sigma  for  high  maturity  organizations. 

•  High  maturity  areas  need  more  clarification. 

•  Develop  training  materials  for  levels  4  and  5.  Provide  an  appraisal  tool  set. 

•  Information  on  levels  4/5  is  lacking. 

How  the  SEI  Is  Addressing  Open  Comments 

To  address  open  comments,  the  SEI  is  doing  the  following: 
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•  Analyzing  the  change  requests  created  from  comments  received  as  part  of  this  project  for 
the  development  of  CMMI  Version  1 .2,  including  CMMI  models  and  CMMI  training 
courses. 

•  Using  comments  to  guide  the  development  of  new  products  to  help  organizations  reach 
maturity  levels  4  and  5. 

•  Planning  a  CMMI  interpretation/implementation  course.  Comments  will  be  used  to  help 
define  the  requirements  for  this  course.  This  course  will  concentrate  on  providing  inter¬ 
pretation  and  implementation  guidance  to  address  many  of  these  comments. 

•  Offering  a  course  called  High  Maturity  with  Statistics.  For  more  information,  see 
http://www.sei.cmu.edu/products/courses/high-maturity-statistics.html.  The  SEI  also  of¬ 
fers  Process  Measurement  and  Analysis  courses.  For  more  information,  see 
http://www.sei.cmu.edu/products/courses/p49.html. 

•  Offering  Personal  Software  Process  (PSP)  and  Team  Software  Process  (TSP)  to  help 
high  maturity  organizations.  These  technologies  address  process  improvement,  including 
implementation  of  the  higher  maturity  practices  that  are  described  in  CMMI.  More  in¬ 
formation  about  PSP  and  TSP  can  be  found  at  http://www.sei.cmu.edu/tsp/. 
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6  Conclusion 


The  interest  in  contributing  to  and  the  results  from  the  Interpretive  Guidance  Project  affirmed  the 
SEI’s  confidence  in  CMMI  as  an  excellent  tool  for  process  integration  and  product  improvement. 
The  valuable  input  from  those  organizations  and  individuals  who  participated  in  this  project  will 
help  guide  the  SEI  in  providing  the  most  appropriate  products  and  services  and  conducting  the 
most  needed  research  that  will  support  the  CMMI  user  community. 


Ongoing  SEI  activities  that  are  addressing  comments  received  by  the  Interpretive  Guidance  Pro¬ 
ject  include  the  following: 

•  developing  SCAMPI  B  and  C  appraisal  methods 

•  collecting  and  publishing  cost  and  benefit  information  about  CMMI  adoption,  including 
ROI  data 

•  maintaining  CMMI  FAQs  on  the  SEI  Web  site 

•  studying  the  needs  of  small  organizations  and  how  CMMI  can  be  applied  successfully  in 
these  organizations 

•  coordinating  contributions  from  CMMI  adopters  to  conferences,  publications,  and  Web 
sites  that  describe  how  they  have  adopted  CMMI 

•  developing  a  single  Introduction  to  CMMI  course  that  addresses  both  staged  and  continu¬ 
ous  representations 

•  planning  for  a  new  intermediate-level  interpretation  course  for  CMMI  adopters 

As  a  result  of  this  project,  interpretive  guidance  will  be  developed  in  the  following  three  areas: 

1 .  helping  organizations  familiar  with  legacy  process  improvement  models  that  wish  to  upgrade 
to  CMMI 

2.  providing  guidance  for  addressing  overlaps  in  CMMI  models,  specifically  the  overlap  be¬ 
tween  generic  practices  and  process  areas 

3.  providing  guidance  for  implementing  generic  practices 

Finally,  the  comments  received  during  the  Interpretive  Guidance  Project  are  an  important  input  to 
the  CMMI  Version  1.2  development  effort.  Of  the  approximately  7,500  total  comments  received, 
approximately  2,700  are  open  and  approximately  2,200  will  be  analyzed  by  the  Version  1.2 
CMMI  Product  Development  Team  to  improve  CMMI  models,  appraisal  methods,  and  training. 
This  input  will  help  to  create  better  products  for  current  and  future  CMMI  users. 
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Appendix  A:  Responses  to  General  Ques¬ 
tions  in  the  Interpretive  Guid¬ 
ance  Questionnaire 


This  appendix  contains  a  summary  of  the  responses  to  general  questions  about  CMMI.  These 
charts  summarize  only  the  responses  from  the  questionnaire  since  BOF  activities  were  less  struc¬ 
tured  than  the  questionnaire  and  did  not  seek  the  answers  to  these  specific  questions  during  each 
event.  The  questions  were  administered  between  May  1,  2003  and  June  1,  2003.  More  informa¬ 
tion  about  the  respondents,  the  responses  themselves  as  well  as  these  figures,  is  available  in  the 
companion  report  CMMI  Interpretive  Guidance  Project:  Preliminary  Report  [Chrissis]. 

Global  Issues  Q1 :  Adequacy  of  CMMI 


In  your  opinion,  is  CMMI  adequate  for  guiding  process 
improvement? 


Didn't  Respond 
Don't  know 
Rarely  if  ever 
Sometimes 

More  often  than  not 
Almost  always 

0%  10%  20%  30%  40%  50% 

Percent  of  Respondents 


Figure  20:  Global  Issues  Q1 :  Adequacy  of  CMMI 
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Global  Issues  Q4a:  Leveraging  Earlier  Investments 


Adopting  CMMI  will  help  us  to  leverage  our  earlier  investments 
in  process  improvement. 


Figure  21:  Global  Issues  Q4a:  Leveraging  Earlier  Investments 

Global  Issues  Q4b:  Adequacy  of  Training,  etc. 


Existing  CMMI  training  courses,  guidance  documents,  Web 
resources,  and  other  process  assets  are  adequate  for  our  purposes. 


Didn't  Respond 
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Strongly  Disagree 
Disagree 
Agree 
Strongly  Agree 
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Figure  22:  Global  Issues  Q4b:  Adequacy  of  Training ,  Etc. 
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Global  Issues  Q4c:  Appraisals 


Figure  23:  Global  Issues  Q4c:  Appraisals 

Global  Issues  Q4d:  Cost 
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Global  Issues  Q4e:  Systems  and  Software  Combined 


Figure  25:  Global  Issues  Q4e:  Systems  and  Software  Combined 

Global  Issues  Q4f:  Mapping  Processes 


Figure  26:  Global  Issues  Q4f:  Mapping  Processes 
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Global  Issues  Q4g:  Tracking  Changes 


We  have  had  difficulty  tracking  the  changes  and  additions  from 
models  that  we  have  previously  used. 


Didn't  Respond 

Don't  Know 

_ 

Strongly  Disagree 

Disagree 

Agree 
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Figure  27:  Global  Issues  Q4g:  Tracking  Changes 

Global  Issues  Q4h:  Two  Representations 


Having  a  choice  between  the  two  model  representations  (staged 
or  continuous)  and  variations  (SW,  SE,  IPPD,  SS)  has  been  helpful  for  us. 


Percent  of  Respondents 


Figure  28:  Global  Issues  Q4h:  Two  Representations 
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Global  Issues  Q5:  Return  on  Investment 


Does  your  organization  need  ROI  or  other  quantitative  evidence 
to  help  make  the  business  case  for  adopting  CMMI? 


Didn't  Respond 

No,  it’s  not  a  real  issue 
for  us 

No,  we've  already  built  a 
good  business  case 

Yes,  it  certainly  would 
help  to  have 

Yes,  we  must  have  it 
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Figure  29:  Global  Issues  Q5:  Return  on  Investment 
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Appendix  B:  Summary  of  Comments  by 

Process  Area  and  Generic 
Practice 


Process  Areas 

Figure  30  illustrates  the  number  of  comments  received  by  CMMI  process  area.  Process  area  ab¬ 
breviations  are  in  Appendix  B  of  any  CMMI  model  found  at 
http://www.sei.cmu.edU/cmmi/models/models.html#models. 

Total  Comments  by  Process  Area 
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Figure  31  illustrates  the  number  of  open  comments  about  each  of  the  CMM1  process  areas. 

Open  Comments  by  Process  Area 


Figure  31:  Number  of  open  comments  by  process  area 
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Generic  Practices 


Figure  32  indicates  the  number  of  comments  received  on  each  of  the  CMMI  generic  practices. 
There  were  50  comments  that  made  general  suggestions  about  CMMI  generic  practices  and  15 
comments  about  generic  practice  common  features. 


Total  Comments  by  Generic  Practice 


Figure  32:  Total  number  of  comments  received  by  generic  practice 
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Figure  33  indicates  the  number  of  open  comments  about  each  of  the  CMMI  generic  practices. 


Open  Comments  by  Generic  Practice 
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Figure  33:  Number  of  open  comments  received  by  generic  practice 
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